Minimum Perfect H Fast N-gram Language

A new technique is proposed for N-gram language model (LM) retrieval based on minimum perfect hashing (MPH). A hierarchical data structure is used to store N-gram scores in hash tables according to the order of N-grams, and a LM score is retrieved by probing the appropriate hash table slot without collision. Both integer key and character-string key based MPH functions are studied. The proposed MPH-based technique for N-gram LM lookup was evaluated on the Switchboard database and compared with the hierarchical binary search method of ISIP and the combined hash and linear search method of HTK. The proposed MPH-based technique outperformed the ISIP and HTK methods in significantly reduced LM retrieval time for bigram and trigram LMs.